This invention relates generally to a method for cache coherency protocol with built in avoidance for conflicting responses in a multi-node system, and more particularly to providing a method, system and computer program product capable of establishing and maintaining cache coherency in a “switchless” distributed shared memory computer system.
Computer systems have developed from a single processor system to a large symmetric multi-processor (SMP) system. FIG. 1 illustrates a conventional node 10 of a symmetric multiprocessing computer system, including a plurality of processors 11-15 with associated cache and directory, interconnected by a shared level of cache 16 with associated directory, a storage 17 shared amongst the system's processors, and common I/O devices 18 interconnected to other nodes within a multi-node system through a plurality of interconnect buses 19. A fetch request targeting a given line entering the system can be initiated by any of the plurality of processors 11-15 that upon missing the processor's private cache will traverse the system in search of a target line address and associated data. Upon the cache miss, the request initially enters the shared level of cache 16, and accesses the cache's directory to determine if the target line exists in the shared level of cache. If the line exists in the shared level of cache, commonly referred to as a directory hit, the processor's request will potentially be satisfied and the target lines data is returned to the requesting processor. However, if a directory miss is encountered or the line exists in the shared level of cache but in a state that does not satisfy the processors request, i.e. a read-only hit when the fetch request is for exclusivity, a request will be launched to remote nodes through the interconnect buses 19, or to the locally attached shared storage 17 of node 10.
Methods for maintaining cache coherency have become a critical design point in large SMP systems. Maintaining coherency across caches located on different nodes is a very complicated task. With every new SMP design, a unique set of complex issues arises, such as issues related to operation stalling, data coherency, or window conditions that require special handling.
In existing large SMP systems, overall system performance has grown dramatically, resulting in additional cache levels being required, and an increase in cache sizes. With the introduction of each new cache levels, maintaining data integrity has become more complex. In order to overcome the complexity issue, system designs include the use of a fully connected topology to allow simpler handling of the cache coherency across multiple nodes and smaller latency penalties in reaching each node. FIG. 2 illustrates a fully connected system topology of a multi-node system 20 including a plurality of interconnect buses 25, connecting a multitude of remote nodes 21-24, which follow a given coherency protocol. Each remote node 21-24 includes the same elements as shown in FIG. 1, providing for a plethora of processors within the multi-node system 20, and a larger aggregate shared level of system cache. Even with the use of a fully connected topology, there are still window conditions which could result in a given operations initially detecting a reject condition on one remote node (21, 22, 23 or 24) while another remote node (21, 22, 23 or 24) indicates that it has the highest coherency point in the system 20 and will be able to process the operation. This can create scenarios that if not addressed may lead to data integrity problems.
It would be desirable to be able to prevent conflict cache state detection across multiple caches in a multi-node system.